Skip to content

Conversation

@rebornplusplus
Copy link
Owner

This is only to test the CI. :)

Rafid Bin Mostofa and others added 27 commits May 13, 2024 10:08
This commit adds support for fetching packages from multiple archives.
It introduces a new field "archives.<archive-name>.priority" which takes
in an integer and specifies the priority of a certain archive. This
value is considered when there are multiple archives. A package is
fetched from the archive with highest priority, unless the slice
definition file of that package specifies a particular archive using the
"archive" field.

TODO:
  a. Unit tests for cmd_cut.go, focusing on the archive selection for
     each package ("selectPkgArchive"). The PR for Chisel DB is moving
     code around and it seems best at the moment to wait for that change
     to be somewhat finalized.

Reference: Specification RK018.
In chisel.yaml, archive definitions can now use the "pro" value to
specify Ubuntu Pro archives. The ``archives.<archive>.pro`` value
currently accepts the following values: "fips", "fips-updates", "apps"
and "infra". Any other values are ignored (with a log message) and reset
to "" (empty).

The following table shows the base repository URL for each of the
supported pro values:

    fips            https://esm.ubuntu.com/fips/ubuntu
    fips-updates    https://esm.ubuntu.com/fips-updates/ubuntu
    apps            https://esm.ubuntu.com/apps/ubuntu
    infra           https://esm.ubuntu.com/infra/ubuntu

The actual mechanism for fetching packages from these archives are not
in the scope of this commit and will follow in a later commit.

While using multiple archives, the "priority" values of these Ubuntu Pro
archives should be consistent with the following table:

    fips            20
    fips-updates    21
    apps            16
    infra           15

Note that chisel does not cross-check nor ensure the consistency of
"priority" with "pro" values.

Reference: Specification RK018.
This commit adds the support for fetching packages from the Ubuntu Pro
archives. As stated in previous commit(s), Chisel will currently support
fetching from the following Ubuntu Pro archives:

    fips            https://esm.ubuntu.com/fips/ubuntu/
    fips-updates    https://esm.ubuntu.com/fips-updates/ubuntu/
    apps            https://esm.ubuntu.com/apps/ubuntu/
    infra           https://esm.ubuntu.com/infra/ubuntu/

By default, Chisel will look for credentials in the
``/etc/apt/auth.conf.d/`` directory, unless the environment variable
``CHISEL_AUTH_DIR`` is set. In which case, it will look for
configuration files in that directory. The configuration files may only
have the ".conf" extensions or no extensions. The format of these
configuration files with further details have been introduced in commit
86bcf61 previously. In short, a configuration file may look like the
following:

    machine esm.ubuntu.com/apps/ubuntu/ login <username> password <password>
    machine esm.ubuntu.com/fips-updates/ubuntu/ login <username> password <password>

Note that any Pro archive whose credentials are not found will be
ignored (with logs).

A sample chisel.yaml with different Pro archives may look like below:

    format: v1

    archives:
      ubuntu:
        priority: 10
        version: 22.04
        suites: [jammy, jammy-updates, jammy-security]
        components: [main, universe]
        public-keys: [ubuntu-archive-key-2018]
        default: true
      ubuntu-fips:
        pro: fips
        priority: 20
        version: 22.04
        suites: [jammy]
        components: [main]
        public-keys: [ubuntu-archive-key-2018, fips-key]
      ubuntu-fips-updates:
        pro: fips-updates
        priority: 21
        version: 22.04
        suites: [jammy-updates]
        components: [main]
        public-keys: [ubuntu-archive-key-2018, fips-key]
      ubuntu-esm-infra:
        pro: infra
        priority: 15
        version: 22.04
        suites: [jammy-infra-updates, jammy-infra-security]
        components: [main]
        public-keys: [ubuntu-archive-key-2018, esm-key]
      ubuntu-esm-apps:
        pro: apps
        priority: 16
        version: 22.04
        suites: [jammy-apps-updates, jammy-apps-security]
        components: [main]
        public-keys: [ubuntu-archive-key-2018, esm-key]

    public-keys:
      ...

Reference: Specification RK018.
This merges the changes in letFunny#12
into this branch.
Export ProValue type and related constants: ProNone, ProFIPS,
ProFIPSUpdates, ProApps, ProInfra. Additionally, raise errors on invalid
"pro" values.
Do not fetch indexes for a particular suite if the suite repository does
not support <arch> architecture packages. The "Architectures" field in
the InRelease/Release file should specify the list of supported
architectures.
To run the real archive tests for Ubuntu Pro archives, the host machine
needs to be Pro-enabled first.

    sudo pro attach <pro-token> --no-auto-enable

There are tests for fips, fips-updates, esm-apps, esm-infra archives.
Thus, the respective services have to be enabled as well to generate the
APT credentials.

    sudo pro enable fips fips-updates esm-apps esm-infra --assume-yes

We need to pass the --pro-archive flag in addition to the --real-archive
flag to run the tests for Pro archives.

    go test ./internal/archive/ --real-archive --pro-archive

Note that the credentials file(s) in /etc/apt/auth.conf.d (or
CHISEL_AUTH_DIR) may require root permission.

test: pass missing pro field in archive.Options
This commit adds support for fetching packages from multiple archives.
It introduces a new field `archives.<archive-name>.priority` which
takes in a signed integer and specifies the priority of a certain
archive. A package is fetched from the archive with highest priority.

It also deprecates the concept of default archive in chisel.yaml.
However, the field can still be present and it will be parsed but
IGNORED.

DEPRECATED: "archives.<archive>.default" field in chisel.yaml.
---------

Co-authored-by: Alberto Carretero <angelalbertoc.r@gmail.com>
Rafid Bin Mostofa and others added 30 commits October 15, 2024 10:59
FIPS is not available in Jammy yet.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants